Edge compute systems and methods

ABSTRACT

An exemplary system includes a cloud server configured to provide a service to a client device by way of a network. The system further includes an edge computing device configured to provide, from an edge of the network to the client device, a latency-sensitive task associated with the service. The cloud server and the edge computing device are configured to operate on distinct and matching datasets to provide the service to the client device.

RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 15/861,461, filed Jan. 3, 2018, and entitled “EdgeCompute Systems and Methods,” which is hereby incorporated by referencein its entirety.

BACKGROUND INFORMATION

A client-server model is commonly employed to execute computingapplications. In such a model, tasks of an application are typicallydistributed between a server and a client in accordance with apredetermined, static partition that is defined prior to runtime. Theclient may perform certain tasks using resources of a client device onwhich the client is installed and may offload the performance of othertasks by requesting that the server perform the other tasks usingresources of a server device on which the server is installed. It iscommon for computationally intensive tasks to be assigned to the serverto utilize the computing resources of the server device, which resourcesare typically greater than the resources of the client device.

However, latency of data communications between the client and theserver in a traditional client-server architecture, such as atraditional cloud-service architecture, may be too great to allowlatency-sensitive tasks of an application to be assigned to the server.In addition, the limited resources of the client device may preclude theclient from performing latency-sensitive tasks that are computationallyintensive. Moreover, scaling a traditional client-server architecture tovirtualize client device compute is linear and therefore not economical.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate various embodiments and are a partof the specification. The illustrated embodiments are merely examplesand do not limit the scope of the disclosure. Throughout the drawings,identical or similar reference numbers designate identical or similarelements.

FIG. 1 illustrates an exemplary edge compute system according toprinciples described herein.

FIG. 2 illustrates an exemplary data-processing architecture accordingto principles described herein.

FIG. 3 illustrates an exemplary configuration of computing tasks of aclient, a speed layer, and a batch layer according to principlesdescribed herein.

FIG. 4 illustrates an exemplary implementation of an edge compute systemaccording to principles described herein.

FIG. 5 illustrates an exemplary edge compute method according toprinciples described herein.

FIG. 6 illustrates an exemplary computing device according to principlesdescribed herein.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Exemplary edge compute systems and methods are described herein. Systemsand methods described herein provide an architecture that provideslow-latency data communications over a network and utilizes thelow-latency data communications and network-based computing resources tooptimize performance of computing tasks of an application executing on aclient device, particularly performance of computing tasks that arecomputationally intensive and/or latency sensitive. For example, systemsand methods described herein may leverage low-latency datacommunications and a distributed data-processing architecture to offloadperformance of select computing tasks of the application to one or moreedge computing devices while performing other select computing tasks ofthe application at the client device. This may allow offloading ofcertain latency-sensitive and/or computationally intensive tasks thatpreviously could not be offloaded from the client device due tolatencies and configurations of conventional data-processingarchitectures.

In certain exemplary systems and methods described herein, a cloudserver may be configured to provide a service to a client device by wayof a network. The cloud server may implement a batch layer of adata-processing architecture. An edge computing device may implement aspeed layer of the data-processing architecture, which speed layer maybe configured to provide one or more latency-sensitive components ortasks associated with the service to the client device in response toone or more requests from the client device to offload the one or morelatency-sensitive components or tasks. The edge computing device may beseparate from and communicatively coupled to the cloud server and may belocated at an edge of the network such that a latency of datacommunications between the edge computing device and the client deviceis lower than a latency of data communications between the cloud serverand the client device. With a speed layer of the data-processingarchitecture implemented at the edge of the network, speed layercomputing tasks associated with a service (e.g., speed layer computingtasks of an application) may be performed at the edge of the networkwhere latency of data communications with the client device aregenerally lower than latency of data communications between the clientdevice and the cloud server. This may improve the speed in which theclient device receives results of offloaded computing tasks whencompared to conventional architectures in which a speed layer and abatch layer are implemented together in a cloud server data center. Theimproved speed may allow the client device to offload computing tasksthat are latency sensitive and/or computationally intensive in a mannerthat may facilitate optimized performance of the application and/orimproved user experience with the application. Examples of suchoptimizations are described herein.

Systems and methods described herein may provide additional oralternative features and benefits as may serve a particularimplementation. Various embodiments will now be described with referenceto the figures. The disclosed embodiments may provide one or more of thefeatures and benefits mentioned above and/or various additional and/oralternative features and benefits that will be made apparent herein.

FIG. 1 illustrates an exemplary edge compute system 100 (“system 100”)configured in accordance with methods and systems described herein. Asshown, system 100 may include, without limitation, a cloud server 102,an edge computing device 104, and a client device 106 communicativelycoupled one to another by way of a network 108. Cloud server 102 andedge computing device 104 may operate in tandem to provide a service 110over network 108 for access by a client 112 implemented by client device106.

Network 108 may include any provider-specific wired or wireless network(e.g., a cable or satellite carrier network or a mobile telephonenetwork), the Internet, the World Wide Web, a wide area network, anyother suitable network, or any combination of such networks. Datacommunications may flow between cloud server 102, edge computing device104, and client device 106 via network 108 using any communicationtechnologies, devices, media, and protocols supportive of datacommunications, including, but not limited to, socket connections,Ethernet, data transmission media, communication devices, TransmissionControl Protocol (“TCP”), Internet Protocol (“IP”), Hypertext TransferProtocol (“HTTP”), HTTPS, Session Initiation Protocol (“SIP”), SimpleObject Access Protocol (“SOAP”), Extensible Mark-up Language (“XML”) andvariations thereof, Real-Time Transport Protocol (“RTP”), User DatagramProtocol (“UDP”), Global System for Mobile Communications (“GSM”)technologies, Code Division Multiple Access (“CDMA”) technologies,Evolution Data Optimized Protocol (“EVDO”), third-generation (“4G”)wireless network technologies, fourth-generation (“4G”) wireless networktechnologies such as 4G Long Term Evolution (“LTE”) technologies,fifth-generation (“5G”) wireless network technologies, Time DivisionMultiple Access (“TDMA”) technologies, radio frequency (“RF”) signalingtechnologies, wireless communication technologies, in-band andout-of-band signaling technologies, and other suitable communicationstechnologies. While only one network 108 is shown to interconnect cloudserver 102, edge computing device 104, and client device 106 in FIG. 1,it will be recognized that cloud server 102, edge computing device 104,and client device 106 may communicate with one another by way of one ormore interconnected networks as may serve a particular implementation.

Network 108 may provide for low-latency data communications with clientdevice 106, including low-latency data communications between edgecomputing device 104 and client device 106. For example, in certainexamples, network 108 may provide latencies between about one and twentymilliseconds, and preferably less than about five milliseconds (e.g.,less than ten milliseconds round trip) for data communications betweenedge computing device 104 and client device 106 and between about twentyand one hundred milliseconds, such as about fifty milliseconds or more(e.g., about one hundred milliseconds or more round trip) for datacommunications between cloud server 102 and client device 106.

In certain implementations, network 108 may include a mobile wirelessnetwork to which client device 106 may connect. To this end, network 108may include appropriately configured components of a mobile wirelessnetwork such as radio access networks, wireless communication basestations, eNodeB elements, central offices, backhaul networks, corenetworks, etc. In certain implementations, network 108 may providelow-latency data communications, such as those described above, byimplementing 5G wireless network technologies.

Components of system 100 may be configured to utilize low latencies ofnetwork 108 to optimize service 110 and/or client 112, such as in any ofthe ways described herein. Such optimizations may result from and/orlead to optimal usage of computing resources and/or improved userexperiences with service 110 and/or client 112. Examples of suchoptimizations are described herein.

Client device 106 may include any computing device configured to performclient-side operations described herein. Client device 106 may include,without limitation, a mobile phone device such as a smartphone device, atablet computer, a personal desktop or laptop computer, a gaming device,a virtual reality device such as a virtual reality headset, an augmentedreality device such as an augmented reality headset, a media playerdevice configured to receive, process, and present media content, and/orany other computing device or combination of computing devicesconfigured to access service 110 over network 108.

Client device 102 may implement client 112, which may include a clientapplication installed and configured to execute on client device 102.Client 112 may be configured to direct client device 102 to perform oneor more of the client-side operations described herein. Client 112 maybe implemented as software, firmware, hardware, or any suitablecombination thereof. Client 112 may be configured to access service 110over network 108 and direct client device 102 to perform operations toprovide a user of client device 102 with a user experience based onservice 110. Client 112 may include a virtual reality client, anaugmented reality client, an artificial intelligence client, a mediaplayer client, a virtual assistant client, and/or any other client thatmay be implemented by client device 106.

Client 112 may include various components and may perform various tasksassociated with the components and/or request performance of varioustasks associated with the components. In certain examples, at least someof the components and associated tasks are latency sensitive and/orcomputationally intensive. As an example, client 112 may include avirtual reality client including a rendering component and associatedtasks for rendering views of a virtual reality world for presentation toa user of client device 106. The rendering component and associatedtasks may be latency sensitive and computationally intensive. As anotherexample, client 112 may include an augmented reality client including acomputer vision object recognition component and associated tasks fordetecting object attributes that trigger generation of augmented realityviews. The image recognition component and associated tasks may belatency sensitive and computationally intensive. As another example,client 112 may include an application having an artificial intelligencecomponent and associated tasks that are latency sensitive andcomputationally intensive. As another example, client 112 may include anapplication (e.g., a drone navigation application, an autonomous drivingapplication, etc.) having a computer vision component and associatedtasks that are latency sensitive and computationally intensive.

A component and/or associated tasks may be latency sensitive in thatclient 112 running on client device 106 may have specific latencyrequirements within which the processing of the tasks must be completedand the results of the performance of the tasks available for use byclient 112 to provide a quality user experience. A component and/orassociated tasks may be computationally intensive in that significantcomputing resources must be allotted to perform the tasks within certaintime constraints for client 112 to provide a quality user experience.For example, client 112 may need to return results within a refresh rate(e.g., about 60 Hz, 90 Hz, or 120 Hz) of a display device being drivenby client device 106, which requirement may be especially important to aquality user experience with a client that displays view in real time ornear real time (e.g., a virtual reality or augmented reality client).

Service 110 may include any service provided by cloud server 102 and/oredge computing device 104 for access by client device 106 over network108. Client 112 executing on client device 106 may request service 110from cloud server 102 and/or edge computing device 104, such as bytransmitting a request (e.g., a query, an instruction, etc.) to cloudserver 102 and/or edge computing device 104. Service 110 may beassociated with client 112 generally or with one or more componentsand/or associated tasks of client 112 specifically. For example, service110 may include one or more components and/or tasks corresponding to oneor more components and/or tasks for client 112. Service 110 may be ofany scope, from a service that includes a single task of client 112, toa service that includes tasks associated with a single component ofclient 112, to a service that includes tasks associated with multiple,or even all components of client 112. Service 110 may include a virtualreality service, an augmented reality service, an artificialintelligence service, a computer vision image recognition service, amedia content service, a content data streaming service, a dataanalytics service, a speech recognition service, a web service, arendering service, a light mapping service, a texture mapping service, a3D model computing service, a component-specific service, atask-specific service, and/or any other service that may be provided bycloud server 102 and/or edge computing device 104 for access by clientdevice 106 over network 108. In certain examples, service 110 mayinclude a virtual reality content streaming service that streams, fromcloud server 102 and/or edge computing device 104, data representativeof virtual reality content for rendering by client device 106. Incertain other examples, service 110 may include an augmented realityobject recognition service that analyzes video data and/orcharacteristics to identify objects represented in video captured byclient device 106.

Cloud server 102 may include a system of one or more servers located inthe cloud (at a cloud level of network 108) and configured to provideservice 110 for access over network 108. For example, cloud server 102may include one or more web servers configured to provide a web servicefor access by client device 106 over network 108.

Edge computing device 104 may include one or more servers (e.g., one ormore servers configured in blade server format) and/or other computingdevices physically located at an edge of network 108, nearer, in termsof latency, to client device 106 than cloud server 102 is to clientdevice 106. Edge computing device 104 may be configured to provideservice 110, or one or more components or tasks of service 110, foraccess over network 108. With edge computing device 104 located at anedge of network 108, data communication latencies between edge computingdevice 104 and client device 106 generally will be lower than datacommunication latencies between cloud server 102 and client device 106,which may allow edge computing device 104 to provide service 110, or oneor more components or tasks of service 110, to client 112 more quicklythan cloud server 102 may do so. In certain examples, edge computingdevice 104 may be configured to provide a latency-sensitive component ortask of service 110 to client device 106 to leverage the lower latencyfor data communications between edge computing device 104 and clientdevice 106.

The edge of network 108, as used herein, may include any location withinnetwork 108 that is nearer, in terms of latency, to client device 106than cloud server 102 is to client device 106. In certain examples, theedge of network 108 may include a location at a base station or centraloffice of a mobile wireless network at which edge computing device 104(e.g., a mobile edge computing device) is implemented. Edge computingdevice 104 may be nearer than cloud server 102 to client device 106, interms of latency, for one or more reasons. For instance, edge computingdevice 104 may be nearer to client device 106 in terms of geographicaldistance and/or network hops, which may cause latency of datacommunications between edge computing device 104 and client device 106to be lower than latency of data communications between cloud server 102and client device 106. For example, in certain implementations, edgecomputing device 104 may be one hop or two hops away from client device106, while cloud server 102 is a greater number of hops away from clientdevice 106 such that latency of data communications between edgecomputing device 104 and client device 106 is lower than latency of datacommunications between cloud server 102 and client device 106.

System 100 may provide an end-to-end service delivery architectureconfigured to provide service 110 to client 112. To this end, cloudserver 102 and edge computing device 104 may be configured to implementa unique, distributed data-processing architecture used to provideservice 110 for access over network 108. As shown in FIG. 1, forexample, cloud server 102 may implement a batch layer 114 and edgecomputing device may implement a speed layer 116 of a distributeddata-processing architecture.

Batch layer 114 may be configured to manage and/or process on a completemaster dataset, which dataset may be an immutable, append-only set ofraw data is some implementations. Batch layer 114 may precompute on thedataset and generate batch views that may be requested by client device106. In certain examples, batch layer 114 may be configured toprecompute arbitrary query functions to generate the batch views. Batchlayer 114 may be configured to maximize accuracy and completeness byprocessing all available data when generating views, which may allowbatch layer 114 to fix errors by computing based on the complete datasetand updating existing views. Batch layer 114 may be configured to use adistributed processing system located within the cloud and capable ofhandling large quantities of data. Batch layer 114 may be configured toperform latency-tolerant manipulation of the dataset.

Speed layer 116 may be configured to process on a dataset (e.g., one ormore data streams) in real time or near real time. To this end, speedlayer 116 may use fast and incremental algorithms to process, withminimal latency, the most recent data to generate real-time views thatmay be requested by client device 106. Speed layer 116 may accommodatelatency-sensitive requests, i.e., requests that are subject to lowlatency requirements. Speed layer 116 may be configured to minimizelatency so that real-time views are made available almost immediatelyafter data is received. By positioning speed layer 116 at edge computingdevice 104 at an edge of network 108 instead of at cloud server 102 withbatch layer 114, speed layer 116 may receive data from client device 106more quickly and make real-time views available to client device 106more quickly than if speed layer 116 was implemented at a same networklevel as batch layer 114 (e.g., together with batch layer 114 in thecloud). In addition, in certain examples, speed layer 116 may facilitateinput and/or output functions and logic that can exist off client device106.

Batch layer 114 and speed layer 116 may be configured to manage andoperate on distinct and matching datasets, such as dataset 118-1 and118-2, respectively, as shown in FIG. 1. To this end, batch layer 114and speed layer 116 may each maintain and process on a distinct datasetand may perform one or more operations to ensure that the datasets matcheach other. For example, batch layer 114 and/or speed layer 116 may beconfigured to exchange, by way of network 108, signals that may be usedto synchronize the distinct datasets.

As an example, speed layer 116 may receive data from client 112 and maysplit the data in order to both process the data in accordance withoperations of speed layer 116 and provide the data to batch layer 114 sothat batch layer 114 may process the data in accordance with operationsof batch layer 114. The processing performed on the data by speed layer116 may result in a state of the dataset maintained by speed layer 116,and the processing of the data by batch layer 114 may result in a stateof the dataset maintained by batch layer 114 that matches the state ofthe dataset maintained by speed layer 116, although the matching stateswill be processed on and/or generated at different times at leastbecause of latency and processing time differences across speed layer116 and batch layer 114. The distinct and matching datasets across batchlayer 114 and speed layer 116 are processed at different points in timebecause batch layer 114 may receive its dataset slower than speed layer116 receives its dataset and/or because batch layer 114 generallyprocesses data slower than speed layer 116. Although the distinct andmatching datasets across batch layer 114 and speed layer 116 may matchat different points in time, the datasets may be referred to assynchronized or mirrored datasets.

Speed layer 116 may communicate any information to batch layer 114 thatmay be used by batch layer 114 to synchronize the distinct datasets. Forexample, speed layer 116 may inform batch layer 114 of operations beingperformed by speed layer 116 on its dataset. Batch layer 114 maysimilarly communicate any information to speed layer 116 that may beused by speed layer 116 to synchronize the distinct datasets. Forexample, batch layer 114 may provide data received at batch layer 114,data generated by batch layer 114, and/or information representative ofprocessing performed by batch layer 114 to speed layer 116 for use byspeed layer 116 to perform operations to synchronize its dataset withthe dataset maintained by batch layer 114.

As another example, batch layer 114 and speed layer 116 may receivedistinct and matching datasets from an external source for processing.To illustrate, in examples in which client 112 includes a virtualreality client, batch layer 114 and speed layer 116 may each receive adata stream (e.g., a virtual reality data stream) for processing. Speedlayer 116 may process on its copy of the data stream to generatereal-time views, and batch layer 114 may process on its copy of the datastream to generate batch views.

Batch layer 114 and speed layer 116 may provide generated views to aserving layer of the architecture, which serving layer may beimplemented at edge computing device 104, cloud server 102, or both edgecomputing device 104 and cloud server 102. The serving layer may beconfigured to host and serve the views to client 112 over network 108.Thus, client 112 may request and access the generated views as neededfrom the serving layer of the architecture. Client 112 may include acompositing layer configured to combine views accessed from the servinglayer to form a view that client device 106 may present to a user ofclient device 106.

An exemplary architecture in which batch layer 114 is implemented bycloud server 102 (at a cloud network level) and speed layer 116 isimplemented by edge computing device 104 (at an edge network level thatis at a different network level than the cloud network level) will nowbe described. FIG. 2 illustrates an exemplary architecture 200 thatincludes batch layer 114, speed layer 116, and a serving layer 202communicatively coupled one to another. Architecture 200, which may bereferred to as a distributed data-processing architecture or adistributed service-delivery architecture, may be any suitabledistributed data-processing architecture designed to handle massivequantities of data by implementing both batch-processing andspeed-processing layers. In architecture 200, batch layer 114 and speedlayer 116 may be implemented at different network levels, such as byimplementing batch layer 114 in the cloud and speed layer 116 at an edgeof network 108. For example, architecture 200 may be a modified lambdaarchitecture in which batch layer 114 and speed layer 116 areimplemented at different network levels, such as a modified lambdaarchitecture in which speed layer 116 is implemented at an edge ofnetwork 108 (e.g., at edge computing device 104) instead of in the cloud(e.g., at cloud server 102) with batch layer 114, such as describedherein.

Serving layer 202 may be implemented in the cloud (e.g., at cloud server102), at the edge of the network (e.g., at edge computing device 104),distributed across the cloud and the edge of the network, or at anyother suitable network location or level accessible by client device106. Serving layer 202 may receive and host views generated by batchlayer 114 and speed layer 116, such as a batch view 204 generated by andreceived from batch layer 114 and a real-time view 206 generated by andreceived from speed layer 116. Serving layer 202 may receive requests(e.g., queries) from client device 106 by way of network 108 and provideviews, such as batch view 204 and/or real-time view 206, to clientdevice 106 in response to the requests. Serving layer 202 may include orutilize any suitable data storage (e.g., database) and servingtechnologies for hosting and providing views to client device 106 andfor determining which views to provide to client device 106.

A view hosted by serving layer 202 may include any set of data generatedby batch layer 114 or speed layer 116. A view may be in any suitableformat, such as a map, a page, an image, or a sub-interface that allowsclient 112 to obtain a state of client 112, such as a state of a worldassociated with client 112. As an example, in implementations in whichclient 112 includes a virtual reality client, a view generated by batchlayer 114 or speed layer 116 may include data representative of a view(e.g., a perspective view) of a virtual reality world from a particularvantage point associated with client 112. As another example, inimplementations in which client 112 includes an augmented realityclient, a view generated by batch layer 114 or speed layer 116 mayinclude data representative of a view of a virtual object to bedisplayed together with a live video camera view captured and displayedby client device 106.

Client 112, speed layer 116, and/or batch layer 114 may include sets ofindividual computing tasks that may be performed by client 112, speedlayer 116, and/or batch layer 114, respectively. FIG. 3 illustrates anexemplary configuration 300 in which client 112 is configured toselectively perform computing tasks selected from a set of defined tasks302 (e.g., tasks 302-1 through 302-K), speed layer 116 is configured toselectively perform computing tasks selected from a set of defined tasks304 (e.g., tasks 304-1 through 304-L), and batch layer 114 is configuredto selectively perform computing tasks selected from a set of definedtasks 306 (e.g., tasks 306-1 through 302-M).

The different sets of tasks in configuration 300 may correspond one toanother in any suitable way. As an example, certain tasks included inthe set of tasks 302 implemented by client 112 may also be included inthe set of tasks 304 implemented by speed layer 116 and/or in the set oftasks 306 implemented by batch layer 114. Accordingly, client 112 mayselectively and/or dynamically choose to perform certain tasks includedin the set of tasks 302 and offload other tasks included in the set oftasks 302 for performance by speed layer 116 or batch layer 114. Asanother example, tasks included in the set of tasks 304 implemented byspeed layer 116 may also be included in the set of tasks 306 implementedby batch layer 114. Accordingly, speed layer 116 may selectively and/ordynamically choose to perform certain tasks included in the set of tasks304 and offload other tasks included in the set of tasks 304 forperformance by batch layer 114. Examples of client 112 and speed layer116 selectively and dynamically choosing which tasks to perform andwhich tasks to offload are described herein.

The computing tasks may be modular and individually executable. Theindividually executable tasks may be executed in parallel, which mayallow different threads to be processed in parallel. For example, client112 may be able to run a rendering thread and a network threadconcurrently by executing appropriate individual tasks in parallel.Client 112 may execute tasks without first having complete information(e.g., without a complete dataset) and without regard to where the tasksare being executed (e.g., without regard to whether a task is beingexecuted by client 112 or by speed layer 116). Thus, client 112 does notneed to receive all information before processing and does not requirepre-compilation before processing. A task may be configured to wait foroutput from another task before completing execution, without client 112having to direct the wait or otherwise control the timing of executionof the task.

Speed layer 116 and/or batch layer 114 may be similarly configured toexecute modular computing tasks individually and in parallel. Speedlayer 116 and/or batch layer 114 may be configured to distribute tasksand/or performance of tasks horizontally across multiple devices at acommon network level. For example, speed layer 116 may distributeperformance of tasks included in set of tasks 304 across multipledevices (e.g., across multiple central processing units (CPUs), graphicsprocessing units (GPUs), network processors (NPUs), blade servers,nodes, etc.) located at the edge of network 108.

Computing tasks may be configured to have any suitable level ofgranularity to support modular, individual, and parallel execution. Incertain examples, tasks may be implemented as granular microservices,which may be implemented as micro-applications that perform specializedfunctions quickly. For instance, a microservice may be configured toperform specific mathematical calculations quickly. In some examples,certain microservices may be implemented as granular micro-kernels thatperform specialized functions very quickly. Such a micro-kernel mayimplement a near-minimum amount of computing code (e.g., software) toperform a very specific function and may allow direct or near-directinteraction with hardware (e.g., a CPU, a GPU, an NPU, etc.). In certainexamples, quick-access micro-kernels may be configured for GPUrendering, computer vision, and/or large-scale mathematic computation.

In certain examples, computing tasks may be mapped to components ofclient 112 and/or service 110. For example, a subset of the tasksimplemented by client 112 may be mapped to a component of client 112and/or service 110. When the component is requested by client 112,client 112 may identify and perform the tasks mapped to the requestedcomponent. Client 112 may perform one or more of the tasks locally usingresources of client device 106 and/or offload one or more of the tasksto speed layer 116 at the edge of network 108 or batch layer 114 in thecloud. Examples of components of client 112 and/or service 110 mayinclude, without limitation, a rendering component, an artificialintelligence component, a video image object recognition component, anaugmented reality component, a network communication component, a speechrecognition component, etc.

With functionality of client 112, speed layer 116, and/or batch layer114 implemented as modular, individually executable tasks (e.g., asmicroservices and/or micro-kernels), system 100 may provide significantflexibility and speed for processing high-throughput data. In addition,the implementation may enable certain optimizations that may improveservice 110, client 112, and/or a user experience with client 112.Examples of such optimizations are described herein.

In certain implementations, components and/or tasks of client 112 orservice 110 may be designated to be processed in certain ways withinsystem 100. For example, a component or a task may be designated to beprocessed by client 112 at client device 106, by speed layer 116, or bybatch layer 114. Client 112, speed layer 116, and batch layer 114 may beconfigured to identify such a designation and process the component ortask accordingly. If the component or task is designated to be processedby client 112, client 112 may direct execution of the component or taskto be performed at client device 106. If the component or task isdesignated to be processed by speed layer 116, client 112 may requestexecution of the component or task by speed layer 116, such as bysending requests to edge computing device 104 over network 108. If thecomponent or task is designated to be processed by batch layer 114,client 112 may request execution of the component or task by batch layer114, such as by sending requests to cloud server 102 over network 108.

In certain examples, a component or a task may be designated to beprocessed in a way to be determined at runtime. For such a designation,client 112, speed layer 116, and/or batch layer 114 may determine, atruntime, how to process the component or task. For instance, client 112may determine, at runtime, whether to process the component or task atclient device 106 or to request that the component or task be processedby speed layer 116.

Designation of a component or task may be made in any suitable way. Forexample, client 112, speed layer 116, and/or batch layer 114 may beconfigured to designate components or tasks based on behavior. Forexample, a rendering component may be designated for processing by speedlayer 116, and a voice chat component may be designated for processingby batch layer 114. In some implementations, certain latency-sensitivecomponents or tasks of client 112 or service 110 (e.g., components ortasks requiring responses faster than about twenty milliseconds) may bedesignated for processing by speed layer 116, and more latency-tolerantcomponents or tasks of client 112 or service 110 may be designated forprocessing by batch layer 114. In certain implementations, for example,client 112, speed layer 116, and batch layer 114 may be configured todesignate components such as a rendering component, a computer visioncomponent, and an artificial intelligence component for processing byspeed layer 116 and components such as an e-commerce component, a logcomponent, and a data analytics component for processing by batch layer114.

As mentioned, in certain implementations, system 100 may be configuredto determine at runtime (e.g., in response to a request that thecomponent or task be performed) how a component or a task will beprocessed. Such a determination may be made in view of a designation ofthe component or task or regardless of a designation of the component ortask. An element of system 100 may be configured to make thedetermination based on predefined factors that are designed tofacilitate the component or task being assigned to the best availableresource for processing (e.g., in order to facilitate optimization ofservice 110 and/or client 112, promote efficient use of resources,and/or provide a quality and/or improved user experience with service110 or client 112). The element of system 100 may be configured toreceive and analyze information about the predefined factors, and todetermine how the component or task will be processed. The predefinedfactors may include, without limitation, current, recent, and/orhistorical latencies of data communications with client device 106(e.g., tracked latencies between client device 106 and edge computingdevice 104 or cloud server 102), conditions of network connections,statuses of resources at client device 106 (e.g., current usage, load,capabilities, and/or availability of resources of client device 106)such as a GPU at client device 106, statuses of resources at speed layer116, latencies of processing and/or internal data communications withinclient device 106 (e.g., latency to send a task to a GPU of clientdevice 106), temperatures of components of client device 106, statusesof other tasks assigned or being assigned for processing by elements ofsystem 100, any other suitable factors, or any combination orsub-combination of such factors.

As an example, during operation of client 112, client 112 may call for atask to be executed. Client 112 may determine that current latenciesbetween client device 106 and edge computing device 104 are below adefined threshold and, in response, may offload performance of the taskby requesting that the task be performed by speed layer 116 (e.g., bysending a task performance request to edge computing device 104). Incertain examples, the defined threshold may correspond to a latency tosend the task to a resource (e.g., a GPU) at client device 106, andclient 112 may determine that when the latency to send the task to speedlayer 116 is not greater than the latency to send the task to a resourceat client device 106, client 112 will request that the task be performedby speed layer 116 instead of at client device 106. As another example,client 112 may determine to send a task to speed layer 116 based on astatus of resources of client device 106, such as when usage ofresources of client device 106 has reached a defined threshold (e.g.,when a temperature of a component of client device 106 has reached adefined threshold temperature). As another example, client 112 maydetermine to process a task at the client device 106 because of acurrent network condition (e.g., current latency is greater than adefined threshold, a connection was temporarily interrupted, etc.). Inthis or a similar manner, client 112 may dynamically select an optimalpath for a component or a task to be performed based on currentlatencies and/or other factors. Speed layer 116 may be similarlyconfigured to determine, at runtime, whether a component or a task willbe processed by speed layer 116 or offloaded (e.g., to batch layer 114)based on one or more predefined factors.

To illustrate one example, with client 112 running on client device 106,a user of client device 106 may point a video camera of client device106 to capture video of a scene that includes a building. Client 112 mayperform video recognition operations, which may include breaking a frameof video camera input into an array of features and sending thosefeatures to speed layer 116. If speed layer 116 has enough informationon the edge of network 108 to do feature recognition, speed layer 116will do feature recognition and send results to client 112 (e.g.,results identifying the building). Alternatively, speed layer 116 mayoffload feature recognition processing to batch layer 114 in the cloud.While the offloaded feature recognition processes are being executed atspeed layer 116 or batch layer 114, client 112 may continue with otherprocessing, such as rendering, doing a base feature analysis to identifyfeatures in frames, and retrieving virtual objects for display toaugment captured video camera frames. This may include client 112retrieving a model of a virtual object in low quality from speed layer116 and starting to render the model of the virtual object while speedlayer 116 obtains higher-quality graphical assets (e.g., textures,lighting information from the camera feed, etc.) and renders andprepares to send the higher-quality graphical assets (e.g., beautifullight maps) to client 112.

By intelligently and dynamically determining how to assign processing ofcomponents and tasks based on predefined factors, elements of system 100may adjust how components and tasks are performed on demand and in amanner that may facilitate efficient use of resources and/or an improveduser experience. Thus, elements of system 100 may determine bestresources (e.g., best paths and best processor slicing per GPU) fromtask to task on an as needed basis, instead of being preprogrammed. Thismay provide potentially unlimited ability to process components andtasks at least because processing of the components and tasks may bedivided up across the network in an intelligent and meaningful way thatfacilitates efficient use of resources and/or an improved userexperience with service 100 or client 112. In certain examples, dynamicassignment of task processing may facilitate specific optimizations ofservice 110, such as improved quality of graphics being delivered toclient 112, for example.

Because of low latency of data communications between client 112implemented at client device 106 and speed layer 116 located at the edgeof network 108, specific latency-sensitive and/or computationallyintensive components or tasks of client 112 may be offloaded from clientdevice 106 to speed layer 116 that could not be offloaded previously inconventional data-processing architectures. The offloaded tasks may beexecuted using shared, accelerated microservices of speed layer 116 atthe edge of network 108, at acceptable latencies. In addition,hardware-based encoding and decoding processes on client 112 and atspeed layer 116 at the edge of network 108 may enable high-speed and/orlow-latency offloading such as loading data into flat buffers andstreaming the flat buffer data through a compression codec such as anH.264 or H.265 codec.

Thus, components and tasks of client 112 that have certain latencysensitivities may be offloaded to speed layer 116, while othercomponents and tasks (e.g., components and tasks having even morestringent latency sensitivities) may be performed locally at clientdevice 106. In certain implementations, for example, certainlatency-sensitive components or tasks of client 112 or service 110 suchas components or tasks requiring responses between about five and twentymilliseconds may be designated for processing by speed layer 116, andmore latency-tolerant components or tasks of client 112 or service 110such as components or tasks requiring responses faster than about fivemilliseconds may be designated for processing on client device 106.These ranges may be adjusted as may fit a particular implementationand/or configuration. For instance, in a configuration in whichlatencies of about one millisecond are provided between edge computingdevice 104 and client device 106, components or tasks requiringresponses between about one and twenty milliseconds may be designatedfor processing by speed layer 116.

In certain implementations, computing components and/or tasks that maybe offloaded to speed layer 116 may include, without limitation,distributed rendering, certain components or tasks of distributedrendering such as light map and/or light field tasks (calculating andproviding a light field), computer vision such as image and objectrecognition and/or tracking tasks, state management such as statemanagement of a virtual reality world (e.g., state management of roomsand/or users of a virtual reality world), procedural generation ofassets such as virtual reality assets, world caching such as virtualreality world caching, screen space processing components or tasks suchas calculations related to determining field of view in a virtualreality world, indirect lighting components or tasks such as ray tracingoperations, physics processing components or tasks such as fluiddynamics operations, any other suitable components or tasks of anapplication executing on client device, or any combination orsub-combination of such components and/or tasks.

In certain implementations, elements of system 100 may be configured toperform optimizations associated with predicting and caching operationsthat may improve a quality of experience with service 100 or client 112.For example, client 112 and/or speed layer 116 may be configured toscale up prediction and loading of views in advance because oflow-latency communications between client device 106 and edge computingdevice 104. This may allow client 112 and/or speed layer 116 to cachemore data (e.g., more graphics data and/or higher quality graphics data)further in advance and/or to run faster such that more data is availableto speed layer 116 and/or client 112 for quick-access processing.

To illustrate, in an example involving virtual reality content, client112 includes a virtual reality client having a rendering component,client 112 and/or speed layer 116 may be configured to use data about avantage point in a virtual world to predict where the vantage point willbe located in the future. For instance, velocity of the vantage pointmay be used to predict where the vantage point will be a fewmilliseconds or seconds in the future. Client 112 and/or speed layer 116may use predicted locations of the vantage point to generate and cacheviews in advance such that the views are available for quick-access byclient 112 as needed. Such views may be used by client 112 and/or speedlayer 116 to minimize or avoid interruptions to continuous presentationof the virtual world (e.g., such as may otherwise occur if a networkconnection is temporarily interrupted), which interruptions mayadversely affect the user experience with the virtual reality client. Incertain examples, if continuous presentation of the virtual world is tobe interrupted for a period of time that extends beyond the cachedframes, client 112 and/or speed layer 116 may be configured tore-project one or more previous frames based on predicted movement(e.g., of the vantage point) to generate frames that may be presentedinstead of the virtual world have a severe interruption like coming to ahalt or crashing.

In certain examples, prediction may be performed in tandem at both aserver level of network 108 (e.g., in the cloud or at the edge ofnetwork 108) and at a client level of network 108 (e.g., at clientdevice 106). Server-level prediction may be relevant when synchronizingmultiple users interacting in the same virtual world, for example. Forinstance, if interactions of a user with the virtual world areinterrupted for some reason, speed layer 116 may use prediction andcaching to prevent or minimize any degradation to the experiences ofother users interacting with the virtual world. If the users are racingvirtual vehicles in a virtual world, for example, and connection to oneuser is interrupted, speed layer 116 may use prediction and caching tocontinue to render the vehicle of the user in a predicted manner for ashort period of time until the connection is reestablished.

Client-level prediction may be relevant when client 112 loses connectionto the server (e.g., to cloud server 102 and/or edge computing device104). In such a situation, client 112 may use prediction and caching tocontinue to render a high-quality experience for the user and avoiding ajarring experience such as the virtual world coming to a halt orcrashing.

In certain examples, elements of system 100 may be configured to performheuristic-based caching over time to improve caching based on historicalpatterns and preferences of users interacting with system 100. Forexample, if many requests are received from a particular geographicarea, elements of system 100 may scale up caching operations for thatarea.

In certain examples, elements of system 100 may be configured to providea graceful falloff procedure in a way that leverages architecture 200.In a virtual reality example, for instance, interaction with a virtualworld may be interrupted to an extent that a graceful falloff procedurewill be performed. For example, if a lost network connection is notreestablished quickly enough, elements of system 100 may perform adefined graceful falloff procedure that is natural within the world andavoids an unnatural, abrupt, or jarring interruption from occurring inthe world. In the graceful falloff procedure, the quality of theexperience may degrade because data available for prediction will fadeaway after user interaction stops, but the virtual world may beprevented from coming to an abrupt halt or crashing. In the example ofvehicles racing in a virtual world, for example, if the connection of auser to network 108 is lost and not reestablished quickly enough, agraceful falloff procedure may be applied to the vehicle of the user,such as by gradually slowing the vehicle to a stop as if the vehicle hadbecome disabled. As another example, if a connection to the edge ofnetwork 108 is lost or the network connection starts degrading such thatlatency increases, processing of rendering tasks may be gracefullyoffloaded from speed layer 116 to client 112. The quality of the imagesmay degrade, but there will still be images presented. As anotherexample, if client device 106 starts to overheat, client 112 maygracefully turn off components or offload components to speed layer 116and/or batch layer 114. For instance, if all ray-tracing is beingperformed on client device 106 and client device 106 starts heating up,client 112 may offload ray-tracing to speed layer 116 and/or batch layer114. One of more of the above-described graceful falloff procedures maybenefit from architecture 200, such as by being able to offload specificand/or more tasks to edge computing device 104 and/or by being able topredict and cache data further ahead such that more data is availablefor graceful falloff and/or recovery from a network interruption.

In certain examples, elements of system 100 may be configured to performoperations to enable adaptive rendering in which different qualitylevels of graphics may be rendered at different times to facilitate aquality user experience with service 110 or client 112. For example,client 112 and/or speed layer 116 may be configured to quickly renderlower-quality graphics first and then scale up to higher-qualitygraphics as calculations become available. To illustrate, a usercontrolling an avatar in a first-person virtual reality experience maydirect the avatar to open a door to a room to reveal a new scene to theavatar. Client 112 and/or speed layer 116 may quickly renderlower-quality graphics of the room first and then scale up the graphics.This may allow the user to see a view of the room immediately instead ofwaiting for higher-quality graphics to become available. In certainimplementations, speed layer 116 may provide a close-by cache ofgraphics at the edge of network 108 to allow fast rendering of a view atlow quality while higher-quality models are computed and/or retrieved.

As another example, client 112 and/or speed layer 116 may degradequality of graphics being presented in order to allow the virtualreality world to continue without crashing. For instance, a connectionto speed layer 116 may be lost or degraded, and rendering may beoffloaded to client 112, which may render lower-quality graphics thanspeed layer 116.

To facilitate adaptive rendering, client 112 and/or speed layer 116 maycache different quality levels of graphics for a virtual world such thatthe various graphics are available for use by client 112 and/or speedlayer 116 as needed to support scaling up or down as required. Withrespect to rendering different quality levels of graphics, speed layer116 may be configured to fail up a level to the cloud (to batch layer114) or down a level to client device 106.

In certain examples, system 100 may provide a virtual reality streamingarchitecture that supports end-to-end streaming delivery of virtualreality content on demand and/or in real time (e.g., without having todownload data for an entire virtual reality world in advance ofrendering). The architecture may enable one or more of the exemplaryoptimizations described herein, as well as additional or alternativeoptimizations that may improve a virtual reality experience.

In certain implementations, speed layer 116 may provide distributedresources (e.g., GPUs) at strategic locations, which may provide ashared, resource accelerated (e.g., a GPU accelerated) compute platformthat can be provided as a service, such as a software as a service(SaaS), an infrastructure as a service (IaaS), and/or a platform as aservice (PaaS) model. To this end, speed layer 116 may be implemented toinclude scripts (e.g., Python script) with engine logic for performingone or more of the speed layer operations described herein. Speed layer116 may include or access one or more resources such as GPUs located atthe edge of network 108. In certain examples, speed layer 116 mayimplement a software layer (e.g., CUDA provided by NVIDIA Corporation)that provides direct access to a GPU's virtual instruction set andparallel computing elements for execution of compute kernels.

In certain examples, client 112 may initiate access to the architectureof system 100 by initiating communications with and authenticating tobatch layer 114, which may perform of one or more authorization andaccess management tasks for this purpose. Batch layer 114 may assignclient 112 to a particular speed layer and/or edge computing device,such as speed layer 116 implemented by edge computing device 104, thatis located proximate to client device 106. Batch layer 114 may informclient 112 of the assignment, and client 112 may being communicatingwith the assigned speed layer. Client 112 may offload tasks to theassigned speed layer and/or to batch layer in any of the ways describedherein. The speed layer may include a server configured to receiverequests from client 112, such as a server (e.g., a NODE JAVASCRIPTserver) for receiving HTTP requests. Client 112 may also send requestsfor views to serving layer 202, which will determine which speed viewsand/or batch views to provide to client 112 in response to the requests.Client 112 may use a compositing layer to combine speed and batch viewsto produce a view for presentation to the user of client device 106.

FIG. 4 illustrates an exemplary implementation 400 of an edge computesystem according to principles described herein. Implementation 400 maybe configured to provide a virtual reality service and/or application incertain examples described herein and/or to provide any other suitableservice and/or application in other examples.

As shown, implementation 400 may include cloud server 102, edgecomputing device 104, and client device 106 communicatively coupled oneto another. Cloud server 102, edge computing device 104, and clientdevice 106 may be configured to provide an architecture for delivery ofa service to client device 106. Edge computing device may implement aspeed layer 402 of the architecture, and cloud server 102 may implementa batch layer 404 and a serving layer 406 of the architecture. Cloudserver 102 may be implemented using any suitable web services platformsuch as AMAZON WEB SERVICES (“AWS”).

Client device 106 may execute a client, such as client 112, and, afterauthenticating to cloud server 102 and receiving information about speedlayer 402 from cloud server 102, send requests to cloud server 102 andedge computing device 104 in associated with execution of the client.Client device 106 may receive responses to the requests (e.g., responsesincluding data and/or views) and process the responses to provide aservice and/or user experience with the client to a user of clientdevice 106.

For a rendering component of the client, client device 106 may sendcommunications to speed layer 402 at edge computing device 104 for useby speed layer 402 to generate real-time views, such as real-time view408, of a virtual reality world. For example, client device 106 may sendinformation about a vantage point of a user of client device 106 andchanges to the vantage point as the user navigates within the virtualreality world.

Speed layer 402 may include a node server 410 configured to receivecommunications from client device 106 and to process the communications.In certain examples, node server 410 may include a run-time environmentfor executing code server-side, such as a NODE.JS JAVASCRIPT runtimeenvironment.

Speed layer 402 may also include scripts with logic 412 that may beperformed to execute on the data received by node server 410. Forexample, scripts with logic 412 may include a set of microservices(e.g., a set of micro-kernels) such as described herein.

Speed layer 402 may also include a GPU interface 414 that providesaccess to GPU resources located at the edge of the network, such as GPUresources (not shown) implemented by edge computing device 104. Incertain examples, GPU interface 414 may include multiple instances ofCUDA by NVIDIA, which may provide a parallel computing platform andapplication programming interface models that allow access toCUDA-enabled GPUs. The CUDA platform may be implemented as a softwarelayer that gives direct access to a GPU's virtual instruction set andparallel computational elements, for the execution of compute kernels,such as the micro-kernels described herein.

The exemplary components of speed layer 402 shown in FIG. 4 may processdata in real-time and near real-time to provide real-time views veryquickly, which real-time views may be provided to serving layer 406.

Speed layer 402 may also split data received from client device 106 andprovide a copy of the received data, as well as any other informationabout operation of speed layer 402, to cloud server 102 for use by batchlayer 404. For example, speed layer 402 may provide data received fromclient device 106 to a stream processing platform (“SPP”) 416implemented by cloud server 102. SPP 416 may provide a high-throughput,low-latency platform for processing real-time data feeds. In certainexamples, SPP 416 may be implemented using APACHE KAFKA.

A dashed line 418 extending from speed layer 402 to cloud server 102represents an interface of speed layer 402 with cloud server 102, whichinterface may be used to exchange data between speed layer 402 and batchlayer 404. In such an implementation, cloud server 102 may include acluster computing framework (“CCF”) 420, which may be configured toprocess large amounts of streaming data quickly. CCF 420 may beconfigured to split the compute of large amounts of data among serversby using suitable clustering processes to act on the servers as oneserver. CCF 420 may treat speed layer 402 as a cluster. CCF 420 mayperform processing to stream real-time views generated by speed layer402 to serving layer 406 when serving layer 406 is implemented in thecloud as in implementation 400. In certain examples, CCF 420 may beimplemented using any suitable cluster-computing framework solution.Real-time views generated by speed layer 402 may be provided to servinglayer 406 through SPP 416 and/or CCF 420 in certain examples, orprovided to serving layer 406 by way of other paths in other examples.

Certain data received and processed by SPP 416 may be provided to batchlayer 404 for batch processing. As shown, batch layer 404 may include anaggregator 422, a data warehouse 424, and a cluster platform (“CP”) 426,which may be configured to perform one or more of the batch layeroperations described herein, including generating batch views for accessby client device 106.

Aggregator 422 may be configured to aggregate large masses of streamingdata being received, and classify and/or perform more or more otherprocesses on the data. In certain examples, aggregator 422 may beimplemented using any suitable service solution for collecting,aggregating, and moving large amount of data (e.g., streaming dataflows).

Data warehouse 424 may store data and/or metadata associated with aservice being provided by cloud server 102. For example, data warehousemay store data and/or metadata that describes virtual object assets,such as virtual objects associated with a virtual reality world. Incertain examples, data warehouse 424 may be implemented using anysuitable web service data storage solution.

CP 426 may be configured to perform processing to generate and providebatch views, such as a batch view 428, to serving layer 406. In certainexamples, CP 426 may be implemented using any suitable managed clusterplatform solution that processes and analyzes vast amount of data.

Serving layer 406 may receive and index batch views generated by batchlayer 404 and real-time views generated by speed layer 402. Servinglayer 406 may be configured to receive requests for views from clientdevice 406, and identify and provide selected views to client device 106in response to the request.

FIG. 5 illustrates an exemplary edge compute method 500. While FIG. 5illustrates exemplary operations according to one embodiment, otherembodiments may omit, add to, reorder, and/or modify any of theoperations shown in FIG. 5. One or more of the operations shown in FIG.5 may be performed by system 100 and/or any component(s) orimplementation(s) thereof.

In operation 502, a cloud service is provided from a cloud server to aclient device by way of a network. Operation 502 may be performed by thecloud server, such as by cloud server 102 and/or batch layer 114implemented by cloud server 102. Operation 502 may be performed in anyof the ways described herein.

In operation 504, a latency-sensitive task of the service is providedfrom an edge computing device to the client device by way of thenetwork. Operation 504 may be performed by the edge computing device,such as by edge computing device 104 and/or speed layer 116 implementedby edge computing device 104. Operation 504 may be performed in any ofthe ways described herein, including in response to a request fromclient 112 to offload the latency-sensitive task of the service fromclient device 106 to speed layer 116 at the edge of network 108.

While certain examples described herein have been described in terms ofa single client and client device accessing services, components, andtasks from cloud server 102 and/or edge computing device 104, this isillustrative only. In certain implementations, multiple clientsimplemented on respective client devices may concurrently accessservices, components, and tasks from cloud server 102 and/or edgecomputing device 104, including components and tasks provided by speedlayer 116 as a service. In such implementations, a one-to-onerelationship between a distinct data-processing architecture and aclient is not required and scaling is not limited to linear scaling.Rather, systems and methods described herein may provide services,components, and tasks as a service in a manner that supports aone-to-many relationship and allows scaling that is more economical thatlinear scaling.

In certain implementations, systems and methods described herein may beused to provide a streaming virtual reality service to end uses of theservice. This may include streaming virtual reality data representativeof one or more persistent 3D virtual reality worlds, which streamed datamay be fundamentally different from virtual reality data representativeof other types of conventional virtual reality worlds in various ways.For example, conventional technologies stream virtual reality worlds byvirtualizing hardware and software needed to process and render thevirtual reality worlds. As such, when a conventional service provides astreaming virtual reality world to a user using this virtualizationparadigm, there may be a virtualization of hardware and software of avirtual reality presentation system (e.g., a gaming PC loaded withsoftware representative of the virtual reality world) that exists withinthe cloud (i.e., exists on one or more servers with which a clientdevice used by the user is communicating by way of a network). Thevirtualized system may be dedicated to providing a virtual reality worldto a particular user and may thus correspond to the client device usedby the user in a one-to-one fashion. For example, user inputs may becommunicated from the client device to the cloud-based virtualizationsystem, and a video stream may be communicated from the virtualizationsystem back down to the client device.

As a result, users experiencing a virtual reality world by way of thisconventional paradigm may be assigned such cloud-based virtualizationsof virtual reality presentations systems, and each system may provide arespective user with virtual reality content in essentially the same waythat a system would if it were localized to the user, rather than in thecloud. For example, even though a particular user may receive virtualreality data in a stream (e.g., from the cloud) rather than from a localvirtual reality presentation system (e.g., a gaming PC owned by theparticular user), the software running on the respective virtualizedsystem assigned to the particular user must be compiled, built, anddistributed to be run and used by the user. Consequently, just as withnon-streamed virtual reality data that is processed and renderedlocally, changing aspects of virtual reality data streamed in thisconventional way (e.g., aspects such as the presence, behavior,functionality, appearance, etc., of objects within the virtual realityworld) may require recompilation, rebuilding, and redistribution of thevirtual reality data loaded onto the virtualized system. At the veryleast, a software patch or the like would need to be distributed andinstalled by each virtualized system streaming the conventional virtualreality world.

In contrast, virtual reality data representative of persistent 3Dvirtual reality worlds may operate using a one-to-many stateful paradigmthat is fundamentally different from the one-to-one paradigm describedabove. Rather than loading software representative of the virtualreality world onto dedicated, one-to-one virtualizations of virtualreality presentation systems as described above, a single systemmaintaining a world state of a persistent 3D virtual reality world(e.g., data representative of which objects are within the world, wherethe objects are disposed within the world, how the objects are movingand/or interacting within the world, etc.) may deliver the same stateinformation to many different clients (e.g., client devices of differentusers) that may perform the processing and rendering on their own and/orby offloading certain components or tasks to edge servers relativelyproximate to the clients as latency limitations may allow. Accordingly,virtual reality data representative of a persistent 3D virtual realityworld may be broadcast or multicast to many client devicessimultaneously to allow many users to jointly experience the persistent3D virtual reality world while only one instance of the world ismaintained (e.g., rather than many different instances of the worldbeing maintained on individual virtualized systems, as described above).Additionally, as these persistent 3D virtual reality worlds arecontinuously streamed, all aspects of the appearance and functionalityof the persistent 3D virtual reality worlds and components therein maybe modular, open, dynamic, and modifiable at run time.

As such, one benefit of persistent 3D virtual reality worlds (e.g.,virtual reality worlds streamed using a stateful, one-to-many paradigmsuch as described above) is that 3D virtual reality worlds may bedynamically changed (e.g., on demand, at run time as the virtual realitydata is streaming) in various ways without the virtual reality datarepresentative of persistent 3D virtual reality worlds being recompiled,rebuilt, or even updated by way of a software patch or the like. Forexample, as the virtual reality data of persistent 3D virtual realityworlds is streaming, a provider of the virtual reality data may be ableto freely add or remove objects from the worlds, add games that may beplayed by users, update or adjust the code on which the world is running(e.g., to add new features or functionality, to fix bugs, etc.), changethe functionality of the worlds (e.g., modify the physics of the worlds,etc.), and/or otherwise dynamically contextualize the worlds in anysuitable way.

Another benefit of persistent 3D virtual reality worlds is that virtualreality data of such worlds may be broadcast to many users at oncewithout requiring dedicated virtualizations of hardware and software tobe assigned and reserved for all of the users. In some examples, variousdifferent persistent 3D virtual reality worlds offered by a provider maybe broadcast to a subscriber base of many users as virtual realityexperience options. As such, users in the subscriber base may instantlyswitch from world to world in an analogous way as users may switchthrough different television channels. Like television channels, various3D virtual reality worlds may persistently exist and be persistentlyprovided to users as options to experience regardless of which worlds(if any) the users actually select and experience. Also analogous totelevision channel surfing, client devices used to instantly switch fromworld to world in this way may not store or receive any data related tothe different persistent 3D virtual reality worlds prior to presentingthe worlds to respective users. All the data needed to process, render,and present a particular persistent 3D virtual reality world to the usermay be provided at run time when the world is selected.

In certain embodiments, one or more of the systems, components, and/orprocesses described herein may be implemented and/or performed by one ormore appropriately configured computing devices. To this end, one ormore of the systems and/or components described above may include or beimplemented by any computer hardware and/or computer-implementedinstructions (e.g., software) embodied on at least one non-transitorycomputer-readable medium configured to perform one or more of theprocesses described herein. In particular, system components may beimplemented on one physical computing device or may be implemented onmore than one physical computing device. Accordingly, system componentsmay include any number of computing devices, and may employ any of anumber of computer operating systems.

In certain embodiments, one or more of the processes described hereinmay be implemented at least in part as instructions embodied in anon-transitory computer-readable medium and executable by one or morecomputing devices. In general, a processor (e.g., a microprocessor, aCPU, a GPU, etc.) receives instructions, from a non-transitorycomputer-readable medium, (e.g., a memory, etc.), and executes thoseinstructions, thereby performing one or more processes, including one ormore of the processes described herein. Such instructions may be storedand/or transmitted using any of a variety of known computer-readablemedia.

A computer-readable medium (also referred to as a processor-readablemedium) includes any non-transitory medium that participates inproviding data (e.g., instructions) that may be read by a computer(e.g., by a processor of a computer). Such a medium may take many forms,including, but not limited to, non-volatile media, and/or volatilemedia. Non-volatile media may include, for example, optical or magneticdisks and other persistent memory. Volatile media may include, forexample, dynamic random-access memory (“DRAM”), which typicallyconstitutes a main memory. Common forms of computer-readable mediainclude, for example, a disk, hard disk, magnetic tape, any othermagnetic medium, a compact disc read-only memory (“CD-ROM”), a digitalvideo disc (“DVD”), any other optical medium, random access memory(“RAM”), programmable read-only memory (“PROM”), electrically erasableprogrammable read-only memory (“EPROM”), FLASH-EEPROM, any other memorychip or cartridge, or any other tangible medium from which a computercan read.

FIG. 6 illustrates an exemplary computing device 600 that may bespecifically configured to perform one or more of the processesdescribed herein. As shown in FIG. 6, computing device 600 may include acommunication interface 602, a processor 604, a storage device 606, aninput/output (“I/O”) module 608, and a graphics card 610 communicativelyconnected via a communication infrastructure 612. While an exemplarycomputing device 600 is shown in FIG. 6, the components illustrated inFIG. 6 are not intended to be limiting. Additional or alternativecomponents may be used in other embodiments. Components of computingdevice 600 shown in FIG. 6 will now be described in additional detail.

Communication interface 602 may be configured to communicate with one ormore computing devices. Examples of communication interface 602 include,without limitation, a wired network interface (such as a networkinterface card), a wireless network interface (such as a wirelessnetwork interface card), a modem, an audio/video connection, and anyother suitable interface.

Processor 604 generally represents any type or form of processing unit(e.g., a CPU) capable of processing data or interpreting, executing,and/or directing execution of one or more of the instructions,processes, and/or operations described herein. Processor 604 may directexecution of operations in accordance with one or more applications 614or other computer-executable instructions such as may be stored instorage device 606 or another computer-readable medium.

Storage device 606 may include one or more data storage media, devices,or configurations and may employ any type, form, and combination of datastorage media and/or device. For example, storage device 606 mayinclude, but is not limited to, a hard drive, network drive, flashdrive, magnetic disc, optical disc, RAM, dynamic RAM, other non-volatileand/or volatile data storage units, or a combination or sub-combinationthereof. Electronic data, including data described herein, may betemporarily and/or permanently stored in storage device 606. Forexample, data representative of one or more executable applications 614configured to direct processor 604 to perform any of the operationsdescribed herein may be stored within storage device 606. In someexamples, data may be arranged in one or more databases residing withinstorage device 606. In certain examples, storage device 606 may maintainsurface data, metadata, data streams, video streams, transport streams,and/or any other data received, generated, managed, maintained, used,and/or transmitted by facilities 106 through 116. The storage facilitymay further include any other data as may be used by facilities 106through 116 to perform one of more of the operations described herein.

I/O module 608 may include one or more I/O modules configured to receiveuser input and provide user output. One or more I/O modules may be usedto receive input for a single virtual reality experience. I/O module 608may include any hardware, firmware, software, or combination thereofsupportive of input and output capabilities. For example, I/O module 608may include hardware and/or software for capturing user input,including, but not limited to, a keyboard or keypad, a touchscreencomponent (e.g., touchscreen display), a receiver (e.g., an RF orinfrared receiver), motion sensors, and/or one or more input buttons.

I/O module 608 may include one or more devices for presenting output toa user, including, but not limited to, a graphics engine, a display(e.g., a display screen), one or more output drivers (e.g., displaydrivers), one or more audio speakers, and one or more audio drivers. Incertain embodiments, I/O module 608 is configured to provide graphicaldata to a display for presentation to a user. The graphical data may berepresentative of one or more graphical user interfaces and/or any othergraphical content as may serve a particular implementation. I/O module608 may be omitted from certain implementations.

In some examples, any of the system components described herein may beimplemented by or within one or more components of computing device 600.For example, one or more applications 614 residing within storage device606 may be configured to direct processor 604 to perform one or moreprocesses or functions associated with cloud server 102, edge computingdevice 104, client device 106, network 108, client 112, batch layer 114,speed layer 116, serving layer 202, and/or other components describedherein. Likewise, any suitable storage facility associated with system100 may be implemented by or within storage device 606. Accordingly,computing device 600 may represent an exemplary implementation of cloudserver 102, edge computing device 104, or client device 106.

Graphics card 610 may include any suitable graphics card (e.g., acommercially available graphics card) having dedicated video decodinghardware (e.g., one or more dedicated video decoders) and a programableGPU 616. Graphics card 610 may include additional components in certainembodiments. Graphics card 610 and/or GPU 616 may be configured toexecute and/or assist processor 604 in executing one or more of theexemplary operations described herein. Graphics card 610 may include anysuitable number of graphics cards and/or GPUs as may suit a particularimplementation.

To the extent the aforementioned embodiments collect, store, and/oremploy personal information provided by individuals, it should beunderstood that such information shall be used in accordance with allapplicable laws concerning protection of personal information.Additionally, the collection, storage, and use of such information maybe subject to consent of the individual to such activity, for example,through well known “opt-in” or “opt-out” processes as may be appropriatefor the situation and type of information. Storage and use of personalinformation may be in an appropriately secure manner reflective of thetype of information, for example, through various encryption andanonymization techniques for particularly sensitive information.

In the preceding description, various exemplary embodiments have beendescribed with reference to the accompanying drawings. It will, however,be evident that various modifications and changes may be made thereto,and additional embodiments may be implemented, without departing fromthe scope of the invention as set forth in the claims that follow. Forexample, certain features of one embodiment described herein may becombined with or substituted for features of another embodimentdescribed herein. The description and drawings are accordingly to beregarded in an illustrative rather than a restrictive sense.

What is claimed is:
 1. A system comprising: a cloud server configured toprovide a service to a client device by way of a network; and an edgecomputing device that is separate from and communicatively coupled tothe cloud server, the edge computing device configured to provide, froman edge of the network, a latency-sensitive task associated with theservice to the client device; wherein the cloud server and the edgecomputing device are configured to operate on distinct and matchingdatasets to provide the service to the client device; wherein the cloudserver is configured to provide a latency-tolerant task associated withthe service to the client device in response to a request sent by theclient device to the cloud server; and wherein the edge computing deviceis configured to provide the latency-sensitive task associated with theservice to the client device in response to a request sent by the clientdevice to the edge computing device.
 2. The system of claim 1, whereinthe cloud server and the edge computing device are configured toexchange, by way of the network, signals for synchronizing the datasets.3. The system of claim 1, wherein the distinct and matching datasetshave matching states at different points in time.
 4. The system of claim1, wherein: the cloud server and the edge computing device each receivea data stream; the edge computing device processes on its copy of thedata stream to generate a first of the distinct and matching datasets;and the cloud server processes on its copy of the data stream togenerate a second of the distinct and matching datasets.
 5. The systemof claim 4, wherein: the first of the distinct and matching datasetsincludes real-time views of the data stream; and the second of thedistinct and matching datasets includes batch views of the data stream.6. The system of claim 5, wherein at least one of the cloud server andthe edge computing device is configured to receive the batch views fromthe cloud server and the real-time views from the edge computing device,and to host the batch views and the real-time views for access by theclient device.
 7. The system of claim 1, wherein the distinct andmatching datasets comprise a first virtual reality data stream receivedby the cloud server and a second virtual reality data stream received bythe edge computing device.
 8. The system of claim 1, wherein the edgecomputing device divides a set of tasks such that a first task includedin the set of tasks is performed by the edge computing device and asecond task included in the set of tasks is offloaded to the cloudserver.
 9. The system of claim 1, further comprising: a clientimplemented by the client device, the client configured to determine,based at least in part on a tracked latency of data communicationsbetween the client device and the edge computing device, whether toperform the latency-sensitive task at the client device or to offloadthe latency-sensitive task to the edge computing device.
 10. An edgecomputing device comprising: a communication interface configured tocommunicate, by way of a network, with a client device and with a cloudserver that provides a service to the client device by way of thenetwork; a processor; and a non-transitory computer-readable mediumstoring instructions executable by the processor to direct the edgecomputing device to: operate on a first dataset that is distinct fromand matches a second dataset operated on by the cloud server to providethe service to the client device; and provide, from an edge of thenetwork, a latency-sensitive task associated with the service to theclient device; wherein the cloud server is configured to provide alatency-tolerant task associated with the service to the client devicein response to a request sent by the client device to the cloud server;and wherein the edge computing device is configured to provide thelatency-sensitive task associated with the service to the client devicein response to a request sent by the client device to the edge computingdevice.
 11. The edge computing device of claim 10, wherein theinstructions are further executable by the processor to direct the edgecomputing device to exchange, with the cloud server by way of thenetwork, signals for synchronizing the first and second datasets. 12.The edge computing device of claim 10, wherein the first and seconddatasets have matching states at different points in time.
 13. The edgecomputing device of claim 10, wherein the instructions are furtherexecutable by the processor to direct the edge computing device todivide a set of tasks such that a first task included in the set oftasks is performed by the edge computing device and a second taskincluded in the set of tasks is offloaded to the cloud server.
 14. Amethod comprising: operating, by an edge computing device located at anedge of a network, on a first dataset that is distinct from and matchesa second dataset operated on by a cloud server to provide a service to aclient device by way of the network; and providing, by the edgecomputing device from the edge of the network to the client device, alatency-sensitive task associated with the service; wherein the cloudserver is configured to provide a latency-tolerant task associated withthe service to the client device in response to a request sent by theclient device to the cloud server; and wherein the edge computing deviceis configured to provide the latency-sensitive task associated with theservice to the client device in response to a request sent by the clientdevice to the edge computing device.
 15. The method of claim 14, furthercomprising exchanging, by the edge computing device with the cloudserver by way of the network, signals for synchronizing the first andsecond datasets.
 16. The method of claim 14, wherein the first andsecond datasets have matching states at different points in time. 17.The method of claim 14, further comprising dividing, by the edgecomputing device, a set of tasks such that a first task included in theset of tasks is performed by the edge computing device and a second taskincluded in the set of tasks is offloaded to the cloud server.
 18. Themethod of claim 14, wherein the distinct and matching datasets comprisea first virtual reality data stream received by the edge computingdevice and a second virtual reality data stream received by the cloudserver.
 19. The method of claim 14, further comprising: receiving, bythe edge computing device and from the client device, a request tooffload the latency-sensitive task from the client device to the edgecomputing device; and providing the latency-sensitive task to the clientdevice in response to the request.
 20. The method of claim 14, embodiedas computer-executable instructions on at least one non-transitorycomputer-readable medium.